Device state estimation under pedestrian motion with swinging limb

ABSTRACT

Systems, methods, devices and computer-readable storage mediums are disclosed for device state estimation under pedestrian motion with swinging limb. In some implementations, a method comprises: determining, by a device, that the device is attached to a swinging limb based on a sensor signal; determining, by the device, a rotational velocity component due to the swinging limb based on the sensor signal and limb parameters; estimating, by the device, device velocity; and determining, by the device, user velocity based on the device velocity and the rotational velocity component.

TECHNICAL FIELD

This disclosure relates generally to dead reckoning (DR) and devicestate estimation.

BACKGROUND

Computing an accurate radio navigation-based position solution inchallenging signal environments such as urban canyons and areas of densefoliage can be difficult. In these challenging signal environments,fewer signals are available, and those signals that are available tendto yield less accurate measurements on a device due to environmentalattenuation. One approach to improving the availability and quality ofposition solutions in challenging signal environments is to combineobservations of radio navigation signals with input from other sensorsor signals that measure some aspect of user or antenna motion between orduring the measurement of radio navigation signals. The additionalinformation improves the position solution by subtracting out antennamotion between epochs of radio navigation measurements, effectivelyallowing multiple epochs of measurements to be statistically combined toreduce error.

One approach to improving the availability and quality of positionsolutions blends measurements of radio navigation signals in a KalmanFilter with numerical integration of accelerometer and/or rate gyroscopemeasurements or the like to correct for antenna motion between epochs.For this approach, the numerical integration component is often calledan inertial navigation system (INS) or DR component. In this approach,the DR component is used to subtract antenna motion between epochs somultiple epochs of radio navigation measurements may be combined.However, because the DR component estimates motion from one epoch to thenext, the DR component accumulates errors over time as that motion iscombined over multiple epochs.

It is desirable in pedestrian motion estimation to minimize accumulatedpedestrian motion errors by making the DR component more accurate. Oneway to do this is to select a motion model that allocates adequatedegrees of freedom to model the object undergoing motion withoutassigning unnecessary degrees of freedom. A typical motion model thatdoes a poor job at this task might assign a constant velocity to a userregardless of the motion activity class (e.g., driving, walking).Assigning a constant velocity will result in poor DR performance when,for example, a walking pedestrian with swinging arms holds the device(and hence its sensors) in hand. In this case, the motion of the deviceis not equal to the motion of the user. In fact, the device may at timesmove twice as fast as the user. At other times, the device may appearstationary while the user is walking.

SUMMARY

Systems, methods, devices and non-transitory, computer-readable storagemediums are disclosed for device state estimation under pedestrianmotion with swinging limb.

In some implementations, a method comprises: determining, by a device,that the device is attached to a swinging limb based on a sensor signal;determining, by the device, a rotational velocity component due to theswinging limb based on the sensor signal and limb parameters;estimating, by the device, device velocity; and determining, by thedevice, user velocity based on the device velocity and the rotationalvelocity component.

In some implementations, a device comprises: one or more sensors; one ormore processors; and memory coupled to the one or more processors andconfigured to store instructions, which, when executed by the one ormore processors, causes the one or more processors to perform operationscomprising: determining that the device is attached to a swinging limbbased on a sensor signal provided by the one or more sensors;determining a rotational velocity component due to the swinging limbbased on the sensor signal and limb parameters; estimating devicevelocity; and determining user velocity based on the device velocity andthe rotational velocity component.

Particular implementations disclosed herein provide one or more of thefollowing advantages. A set of techniques are disclosed that robustlyestimate when a user is swinging a device (e.g., in hand or pantspocket) and the state of the user's arm or leg relative to the user'sbody while swinging. The set of techniques results in a DR componentthat is more accurate, which in turn results in a more accurate devicestate estimate (e.g., position, velocity).

The details of the disclosed implementations are set forth in theaccompanying drawings and the description below. Other features, objectsand advantages are apparent from the description, drawings and claims.

DESCRIPTION OF DRAWINGS

FIG. 1 illustrates periodic motion of a hand held device, according toan implementation.

FIG. 2 is a block diagram of an example system for estimating devicestate, according to an implementation.

FIG. 3 is a block diagram of the example update detector shown in FIG.2, according to an implementation.

FIG. 4 is a block diagram of the example update generator shown in FIG.2, according to an implementation.

FIG. 5 is a block diagram of the example predictor shown in FIG. 2,according to an implementation.

FIG. 6 is a block diagram of the example motion model shown in FIG. 5,according to an implementation.

FIG. 7 is a flow diagram of an example process for device stateestimation under periodic motion, according to an implementation.

FIG. 8 is a block diagram of example device architecture forimplementing the features and processes described in reference to FIGS.1-7.

The same reference symbol used in various drawings indicates likeelements.

DETAILED DESCRIPTION

The implementations disclosed herein robustly estimate a device stateunder pedestrian motion with a swinging limb. The disclosedimplementations determine that the user is swinging a device in hand (orin a leg pocket) by detecting periodicity in one or more sensor signalsand knowledge of the orientation of the device in a reference coordinateframe (e.g., geodetic, local level coordinate frame). The state of theuser's swinging limb relative to their body is identified throughknowledge of the device orientation and basic tenets of pedestrianmotion. A DR component models the motion of the user and the motion ofthe user's swinging limb to determine the velocity of the user in thereference coordinate frame.

FIG. 1 illustrates periodic motion of a hand-held device, according toan implementation. In the example shown mobile device 102 (e.g., a smartphone, wrist watch) is held or worn by user 100. The magnitude ormodulus of an acceleration signal provided by accelerometers of mobiledevice 102 tends to be periodic when user 100 walks with mobile device102 in hand because walking is a fundamentally periodic activity. Ifuser 100 is swinging her arm along arm swing path 104, an interestingperiodicity feature is the local minimum acceleration magnitude, wheremobile device 102 is undergoing the least acceleration of the entiremotion cycle. The local minimum acceleration magnitude occurs twice perarm swing motion: once when the arm swings to apex 106 in front of user100 and once when the arm of user 100 swings to apex 108 behind user100. At apexes 106, 108 (the terminus points of arm swing path 104), theestimated speed of the device will be approximately equal to the speedof the user's body. Accordingly, the times when apexes 106, 108 of armswing path 104 are reached provide desirable times to apply pedestrianmotion constraints (e.g., stride length) or device orientationconstraints because small errors in predicting at this time do notresult in significant changes of device orientation or motion. Thisyields a more stable DR component for pedestrian motion, resulting insignificant improvement to the device state estimation.

Example System

FIG. 2 is a block diagram of an example system for estimating devicestate, according to an implementation. In this example implementation,system 200 can include sensors 202, update detector 204, updategenerator 206, predictor 208 and initializer 210.

Sensors 202 can include accelerometers, rate gyroscopes, barometers,magnetometers and the like. Sensors 202 provide sensor signals to updatedetector 204 and update generator 206. Update detector 204 detects andtracks periodicity in the sensor signals and triggers or schedulespedestrian periodic motion constraint updates in predictor 208, asdescribed in reference to FIG. 3. Update generator 206 providesparameters to predictor 208 that are needed to apply pedestrian periodicconstraint updates, and also provides observable inputs for device stateupdates, as described in reference to FIG. 4.

Initializer 210 provides initialization data to predictor 208 to ensurea good initial condition within a linearization boundary of a correctsolution. In some implementations, position and velocity measurementsfrom Global Navigation Satellite System (GNSS) and/or WiFi access point(AP) data can be used to generate initial conditions for position andvelocity. An initial condition for device orientation or attitude can bederived by least squares fitting a device attitude quaternion toreference vector observations using, for example, a Wahba problemsolver, such as the well-known q-method. The q-method can use a varietyof reference vector observations in Earth-fixed and device-fixedreference frames and generate, for example, anEarth-Centered-Earth-Fixed (ECEF) to device quaternion or rotationmatrix (e.g., direction cosine matrix). Some example measurements/modelsinclude but are not limited to: Earth gravity model (Earth-fixed),gravity vector from accelerometer (device-fixed), GNSS data(Earth-fixed), principal component of accelerometer data (device-fixed),International Geographic Reference Field (IGRF) data (Earth-fixed) andmagnetometer data (device-fixed).

In some implementations, predictor 208 generates an estimate of a devicestate, including XYZ device position, XYZ device velocity, deviceattitude using a 4-element quaternion, XYZ device rate gyro biases andXYZ device accelerometer biases. These 16 states can be estimated using,for example, a Kalman Filter formulation, which is updated withpedestrian periodic motion constraint updates, as described in referenceto FIGS. 4-6.

FIG. 3 is a block diagram of the example update detector 204 shown inFIG. 2, according to an implementation. In this example implementation,the sensor signal is an acceleration vector provided by a three-axisaccelerometer. Other sensor signals can also be used such as a rategyroscope signal, which can, for example, be used to determine theperiodic rotation of the arm.

In some implementations, update detector 204 can include periodicitydetector 301, fundamental motion frequency estimator 302, periodicityextractor 304 and scheduler 308. Periodicity detector 301 receives oneor more sensor signals and step cadence frequencies. For example,detector 301 can receive an acceleration vector or magnitude. If anacceleration vector is provided detector 301 can calculate the norm ofthe vector to obtain the magnitude or modulus. In some implementations,detector 301 can pre-process the sensor signal by applying a low pass orband pass filter to remove unwanted frequencies outside the range offrequencies of interest determined by the step cadence frequencies(e.g., 0.45-3.15 Hz). In some implementations, a moving average ormedian filter can be applied to the sensor signals. In someimplementations, a non-causal finite window Gaussian filter can beapplied to the sensor signals.

In some implementations, detector 301 can include a sliding windowfrequency transform (e.g., Discrete Fourier Transform (DFT)) that isapplied to a set of samples of the sensor signal to detect periodicityin the sensor signal. Frequency data generated by detector 301 isprovided to fundamental motion frequency estimator 302, which determinesthe instantaneous fundamental motion frequency of the sensor signal. Forexample, if the length of the sliding window used in detector 301 is atleast as long as several fundamental periods of the user's arm swing,the lowest statistically-non-zero frequency bin output by the frequencytransform can be selected as the fundamental motion frequency. In someimplementations, an instantaneous fundamental motion frequency can alsobe determined as a weighted average over significantly non-zero binfrequencies. In some implementations, the frequency transform can beimplemented using a Fast Fourier Transform (FFT).

The instantaneous fundamental motion frequency can be provided as inputto periodicity feature extractor 304, which extracts a periodicityfeature from one or more segments of the sensor signal that are eachapproximately equal in duration to the fundamental motion period of theobserved periodic sensor signal. In this example, where the sensorsignal is the acceleration magnitude, the periodicity feature is thelocal minimum acceleration magnitude. The local minimum accelerationmagnitude occurs at apexes 106, 108, as described in reference toFIG. 1. If the fundamental motion frequency is f_(s), then thefundamental period of the signal is T_(s)=1/f_(s) and the local minimumacceleration magnitude occurs every T_(s) seconds. There can be a numberof local acceleration minima. For example, a pedestrian with a deviceswinging in his hand tends to reach a local acceleration minimum eachtime one of the user's feet is as far off the ground as possible.Sometimes the strongest frequency is the user's step frequency (e.g.,the frequency of the user's footfalls) and sometimes it is the user'sgait frequency (e.g., the frequency of the user's left footfalls).

In some implementations, the observed periodic signal (e.g.,acceleration magnitude) need not be periodic at all times and thefundamental motion frequency may change over time. In these cases, aphase tracking algorithm (e.g., a phase-locked loop (PLL), delay-lockedloop (DLL), Kalman Filter) can use the fundamental motion frequency totrack the phase of the sensor signal so that the periodicity feature canbe extracted from the sensor signal.

Scheduler 303 is informed each time the periodicity feature is extractedfrom the sensor signal. Scheduler 303 is configured to send a periodicmotion detection (PMD) signal to update generator 206 and a triggersignal/schedule to predictor 208. The trigger signal and/or a schedule(e.g., a predicted time of the next periodicity feature extraction) canbe used to trigger an update in predictor 208. For example, withknowledge of the most recent arm swing apex and the fundamental motionfrequency (arm swing motion frequency), the time of the next apex can bepredicted and used to schedule pedestrian periodic motion constraintupdates. Accordingly, pedestrian periodic motion updates can begenerated by update generator 206 and applied by predictor 208 each timethe arm reaches an apex. At each apex 106, 108, the rotationalaccelerations are substantially zero and the linear velocity of the bodyof user 100 becomes the dominant force detected by the accelerometers.Therefore, the trigger/schedule signal indicates an optimum time toapply pedestrian periodic motion constraints to ensure that thepedestrian periodic motion constraint updates will not introduceunintended errors to the estimated device state provided by predictor208.

The PMD signal can be used by update generator 206 to select update databased on whether or not periodic motion is detected by update detector204, as described in reference to FIG. 4.

In some implementations, fundamental motion frequency estimator 302provides a motion activity class signal, which can be used to determinethat the user is walking and swinging a limb. For example, theperiodicity detector can be run on both accelerometer data and rategyroscope data to determine whether, for example, a pitch component hasa stronger periodicity (e.g., more energy in the frequency domain) thana vertical acceleration component, as described in Xiao et al., “Robustpedestrian dead reckoning (R-PDR) for arbitrary mobile deviceplacement,” 2014 International Conference on Indoor Positioning andIndoor Navigation, (27-30 Oct. 2014), p. 7. Upon such determination, apedestrian motion model that models arm swing motion can be invoked, asdescribed in reference to FIGS. 5 and 6.

FIG. 4 is a block diagram of the example update generator 206 shown inFIG. 2, according to an implementation. In this example implementation,update generator 206 can include update selector 401, pedometer 402,coordinate transform 403, gravity corrector 404, integrators 405, 406and PCA module 407.

In some implementations, pedometer 402 receives an acceleration datavector from 3-axis accelerometers. The norm of the acceleration vector<a_(x), a_(y), a_(z)> can be calculated to generate an accelerationmodulus or magnitude. For example, the magnitude a_(mag) of anacceleration vector

=<a_(x), a_(y), a_(z)> provided by a 3-axis (x, y, z axes)accelerometers is given by,a _(mag)=|

|=√{square root over (a _(x) ² +a _(y) ² +a _(z) ²)}.  [1]

The acceleration magnitude can be used by pedometer 402 to determineuser step frequency and speed s, as described in, for example, U.S.Patent Publication No. US 2013/0085677A1, for “Techniques for ImprovedPedometer Readings,” filed Sep. 30, 2011, which patent publication isincorporated by reference herein in its entirety. In someimplementations, a device rotation rate vector <ω_(x), ω_(y), ω_(z)>from a 3-axis rate gyroscope on the device can be input to pedometer402. The device rotation rate vector <ω_(x), ω_(y), ω_(z)> can be used,for example, in step detection in addition to the acceleration vector.

In some implementations, update generator 206 sends pedestrian periodicmotion constraint updates to predictor 208 at a specified frequency(e.g., 1 Hz) or when no periodic motion is detected, as determined by,for example, the PMD signal provided by update detector 204. Forexample, if no periodic motion is detected, then updates can include achange in altitude Δh across a set period of time (e.g., user gaitperiod). The change in altitude can be determined by a change inbarometric pressure determined by a barometer sensor. The change invelocity Δv can be calculated from the acceleration vector after theacceleration vector has been transformed from a device coordinate frameto a reference coordinate frame (e.g., ECEF or geodetic, local level) bycoordinate transform 403, corrected for gravity by gravity corrector 404and integrated over the last set period of time by integrator 405. Insome implementations, coordinate transform 403 is implemented by aquaternion using techniques known in the art.

In some implementations, update generator 206 sends pedestrian periodicmotion constraint updates to predictor 208 at a specified frequency(e.g., 1 Hz) or when periodic motion is detected, as determined by, forexample, the PMD signal provided by update detector 204. For example, ifperiodic motion is detected, then pedestrian periodic motion constraintupdates include a change in horizontal position Δx across the specifiedtime period, a change in altitude Δh across the specified time period, achange in velocity Δv across the specified time period and a primarydirection ψ of user motion across the specified time period. The primarydirection ψ can be generated by, for example, PCA module 407 usingprincipal component analysis on the acceleration vector.

For example, accelerometers on the device can report accelerations thatare a function of user acceleration, gravity, some minor effects (e.g.,centripetal and Coriolis acceleration due to the rotation of the Earth)and noise. In some implementations, a set of acceleration measurementscan be used to determine a direction of travel by: 1) collecting a setof device acceleration measurements over a specified time interval(e.g., a pre-set time window or the last step period or the last gaitperiod); 2) determining the sum of nearly-constant externalaccelerations (gravity, centripetal, Coriolis) by, e.g., averagingacceleration measurements over several periods; 3) removing thenearly-constant external accelerations by subtracting them from eachmeasurement; 4) computing the sample covariance for the residualgravity-subtracted accelerations computed in step 3; and 5) computing asingular value decomposition of the sample covariance computed in step4. The singular vector with the largest corresponding singular value isthe direction of greatest non-gravity acceleration over the period andis a good measurement of user direction of travel.

In some implementations, Δx can be generated by integrating an averageuser speed determined by pedometer 402. The average user speed can bedetermined by, for example, multiplying an average step frequency by thestride length of the user.

In some implementations, update selector 401 implements a softwareswitch that selects the appropriate motion constraint updates to send topredictor 208 based on the PMD signal received from update detector 204.

FIG. 5 is a block diagram of the example predictor 208 shown in FIG. 2,according to an implementation. In this example implementation,predictor 208 can include motion model 502, estimator 504 andinitializer 210.

In some implementations, motion model 502 implements a path-coordinateformulation for user acceleration given by,

=s·(

_(user)×

_(v))+{dot over (s)}·

_(v),  [2]where s is the speed of the user,

$\overset{.}{s} = {\frac{ds}{dt}.}$is the rate of change of user speed,

_(user) is the rotation rate of the user and

_(v) is a unit tangent vector to a path that the user is travelingalong. The user rotation rate

_(user) can be derived from device rotation rate, which can be obtainedfrom rate gyroscopes on the device. The unit tangent vector

_(v) and user speed s can be obtained from estimator 504. The useracceleration

can be used as an input to estimator 504 as an update to, for example,correct for antenna motion between measurement time epochs. In oneimplementation, the user acceleration in equation [2] is input to anumerical integrator to predict how the user has moved from onemeasurement time epoch to the next. In one implementation, this isimplemented as part of the prediction step of estimator 504.

In some implementations, the rate of change of user speed

$\overset{.}{s} = \frac{ds}{dt}$can be driven by a small white noise process in estimator 504. When theuser and device share an approximately rigid connection, the humanpreference for low acceleration puts limits on the rate of change ofuser speed

$\overset{.}{s} = {\frac{ds}{dt}.}$For example, in many use cases where a device-fixed assumption holds,

$\overset{.}{s} = {\frac{ds}{dt} \approx 0.}$If

${\overset{.}{s} = {\frac{ds}{dt} \approx 0}},$all of the user motion can be predicted with just the user rotation rate

_(user), which yields significant improvements in the DR componentperformance over 6 degrees of freedom (6-DOF) formulations that requiremore accurate accelerometers. The path-coordinate formulation ofequation [2] also yields significant improvements in DR componentperformance over constant velocity models, which cannot predict changesin direction. Some example use cases where the device-fixed assumptionholds includes when the user is driving and the device is docked, whenthe user is walking while looking at the screen of the device or whenthe device is in the user's back pants pocket.

In some implementations, pedestrian motion model 502 receives a motionactivity class signal indicating a motion class as described inreference to FIG. 2. If the motion activity class signal indicates thatthe user is swinging a limb attached to the device, then pedestrianmotion model 502 generates user velocity

_(user) given by

_(device)=

_(user)+

_(rotation)=

_(user)+(

_(L)×

)  [3]where

_(L) is a limb rotation rate vector and

is a limb position vector with an origin at the center of rotation ofthe limb (e.g., a shoulder or hip) to the device.

In equation [3],

_(user) is the desired user velocity and

_(device) is observed via, for example, GNSS data. In someimplementations,

_(L) can be determined from the device rotation rate

_(device) measured directly via rate gyroscopes, where it is observedthat the periodic component of the device rotation rate sensed by therate gyros is substantially due to the swinging limb. Knowledge ofdevice orientation can be determined by, for example, a magnetometervector

and gravitational vector

. The device orientation can then be used to transform

_(L) (e.g., device coordinates) into the same reference coordinate frameas

_(user) and

_(device) (e.g., geodetic, local level coordinates).

To form the limb position vector

a sensed acceleration vector (with gravity subtracted) can be used toidentify the extreme limb directions, which occur at the beginning andend (terminus points) of the limb swing, as described in reference toFIG. 1. In some implementations, the exact limb direction can bedetermined by interpolating between the terminus points, as described inreference to FIG. 6. The limb length ∥

∥ may be entered by the user, set as a default based on averagephysiological features of humans or estimated in estimator 504. With thelimb parameters known, the DR component can be equipped with a moreaccurate model of device motion that separates

_(device) into a user velocity

_(user) component and a rotational component

_(rotation)=(

_(L)×

) due to a swinging limb. Solving equation [3] for user velocity

_(user) gives,

_(user)=

_(device)−

_(rotation)=

_(device)−(

_(L)×

)  [4]

The user velocity of equation [4] can be input into estimator 504 as anupdate to, for example, correct for antenna motion between measurementtime epochs.

In some implementations, estimator 504 is implemented as a KalmanFilter. The state vector can include xyz device position, xyz devicevelocity, device attitude using a 4-element quaternion describing devicerotation from reference coordinate axes to device coordinate axes, xyzdevice rate gyro biases and xyz device accelerometer biases. In someimplementations, the observables can include GNSS and WiFi data. GNSSdata (e.g., latitude, longitude) can include data from any GNSSincluding but not limited to the Global Position System (GPS). WiFi datacan be provided by a positioning service that provides AP data (e.g.,locations of beacons, routers). In some implementations, the uservelocity can be included in the Kalman filter state vector. In such aformulation, the measured user velocity of equation [4] is compared tothe estimated user velocity in the Kalman filter measurement step.

An example Kalman Filter that uses GNSS and WiFi observables and motioncontext is described in U.S. Pat. No. 8,615,253 for “State EstimationUsing Motion Context and Multiple Observation Types,” issued Dec. 24,2013, which patent is incorporated by reference herein in its entirety.In some implementations, the position states are represented in ageodetic coordinate frame and the velocity states are represented in ageodetic, local-level coordinate frame, such as North-East-Up (NEU).Accordingly, in some implementations pedestrian periodic motionconstraint updates may have to be transformed from device coordinateframe to one or more different reference coordinate frames before theycan be applied in the estimator 504.

In some implementations, constraints are incorporated into the Kalmanfilter as measurement updates of the form 0=h(x)+w, where x is thecurrent state estimate, h(x) is the possibly-nonlinear function thatdescribes the constraint space and w is the tolerance variable whosenoise statistics (e.g., measurement noise covariance matrix R) describethe statistical degree to which constraint violation is tolerated. Forexample, to constrain vertical velocity to zero, h(x) would be afunction that projects the current velocity estimate onto the localvertical direction and the square root of the diagonal of the R matrixwould be the expected 1-sigma violation of that constraint. Otherconstraint types would have different h(x) functions and different Rvalues.

In some implementations, the path-coordinate acceleration of equation[2] is used as part of the prediction step of the Kalman filter. Duringa prediction step, the acceleration is input to a numerical integratorto describe the change in the velocity and position estimates from onetime to the next. As such, the process noise matrix Q can be set withnoise parameters for the path-coordinate acceleration model, which mightinclude, for example, noise terms that describe rate gyro andaccelerometer measurement noise and also typical statistics for changesin user speed.

In some implementations, changes in ∥

∥ can be modeled as a percentage or fraction of the average arm (or leg)length (e.g., ¾ the average arm/leg length), which models a majority ofthe velocity contribution due to the limb swing. If the user is walking,∥

∥ can be modeled as the average arm length (most users keep their armsmore or less straight while walking). If the user is running, ∥

∥ can be modeled as

$\frac{\sqrt{2}}{2}.$average arm length (most users jog with their arms bent roughly at 90°).In some implementations, walking or jogging can be detected by, forexample, a difference in step frequencies or acceleration magnitudes. Insome implementations, ∥

∥ can be estimated in the Kalman filter. In some implementations, ∥

∥ can be determined from accelerometer and rate gyro data. For example,the centripetal acceleration due to rotational motion is ω²·∥

∥ where co can be provided by rate gyros. The centripetal part of theacceleration as measured by the accelerometers can be isolated bysubtracting out gravity and function fitting on the residual.

FIG. 6 is a block diagram of the example motion model shown in FIG. 5,according to an implementation.

In some implementations, pedestrian motion model 502 can include limbdirection detector 602 and rotation rate processor 604. Sensedacceleration vector

_(sensed) is input into limb direction detector 602. Sensed accelerationvector is, for example, an acceleration vector formed from the outputsof accelerometers (with gravity subtracted out). Limb direction detector602 determines extreme limb directions in the sensed acceleration data,which occur at the terminus points in the swing path of the limb (SeeFIG. 1). Limb direction detector 602 can use interpolation techniques todetermine a unit vector

_(p) that represents the actual limb direction. This unit vector ismultiplied by the length of the limb ∥

∥ to provide the limb position vector

=∥

∥·

_(p). In some implementations, linear or spherical linear interpolationis used to interpolate between the terminus points. A cross product ofthe limb rotation rate vector

_(L) and the limb position vector

is calculated and subtracted from the device velocity

_(device) to give the user velocity

_(user).

In some implementations, the user rotation rate vector

_(user) can be determined by subtracting out the rotation rate of thelimb

_(L) by, for example, subtracting the periodic component from themeasured device rotation rate

_(device), since the limb rotation rate in most cases only introducesperiodic motion and is typically the largest source of periodic motion.The subtraction can be performed in the time domain by, for example,applying a bandpass filter to

_(device) to allow only signal components that are periodic withfrequencies related to the swinging limb. The remaining measuredrotation after subtracting the periodic component is substantially theuser rotation rate

_(user), as if it were measured by a body-fixed device.

In some implementations, interpolation to determine

_(p) can be performed as follows. First, determine the fundamentalfrequency of periodicity of the limb swing in rate gyro data. Since themotion is periodic, the device will necessarily return to its originalorientation (i.e. undergo no net rotation) if the rate gyros arenumerically integrated over one fundamental period. Assume all therotation is due to limb swing and planar, as if the limb were apendulum. Next, integrate the magnitude of rotation over one fundamentalperiod, noting the maximum and minimum angles, Δθ_(max) and Δθ_(min)relative to the start of the numerical integration. Assume maximumrotation away from vertical (rotation amplitude A) is the same in theforward and backward swing directions and equal to half of(Δθ_(max)−Δθ_(min)). Model the direction of the limb as the direction ofthe gravity vector rotated toward the current direction of user travelby an angle θ(t), where θ(t)=A·sin(f_(o)·t−t_(o)) and t_(o) is some timein the past at which the limb was previously vertical (i.e. was roughlyhalfway in between extreme limb directions).

Example Process

FIG. 7 is a flow diagram of an example process 700 for device stateestimation under pedestrian motion with swinging limb, according to animplementation. Process 700 can be implemented by mobile devicearchitecture 800, as described in reference to FIG. 8.

In some implementations, process 700 can begin by determining that theuser is swinging a limb attached to the device (702). For example, amotion activity class signal can be generated based on analysis ofperiodicity strength (energy levels in the frequency domain) inaccelerometer and rate gyroscope data to determine that the user iswalking and swinging a limb attached to the device, as described inreference to FIG. 3.

Process 700 can continue by determining a rotational velocity component(704). For example, a rotation velocity component due to limb swing canbe determined from (

_(L)×

), where

_(L) is a limb rotation rate vector and

is a limb position vector from the center of rotation of the limb (e.g.,a shoulder or hip) to the device, as described in reference to FIG. 5.

Process 700 can continue by determining a device velocity (706). Forexample, device velocity can be obtained from a device state estimator,as described in reference to FIGS. 5 and 6.

Process 700 can continue by determining user velocity based on thedevice velocity and the rotation velocity due to the swinging limb(708). For example, the user velocity can be determined according toequation [4], as described in reference to FIG. 5.

Example Device Architecture

FIG. 8 is a block diagram of example device architecture 800 forimplementing the features and processes described in reference to FIGS.1-7. Architecture 800 may be implemented in any mobile device forgenerating the features and processes described in reference to FIGS.1-7, including but not limited to smart phones and wearable computers(e.g., smart watches, fitness bands). Architecture 800 may includememory interface 802, data processor(s), image processor(s) or centralprocessing unit(s) 804, and peripherals interface 806. Memory interface802, processor(s) 804 or peripherals interface 806 may be separatecomponents or may be integrated in one or more integrated circuits. Oneor more communication buses or signal lines may couple the variouscomponents.

Sensors, devices, and subsystems may be coupled to peripherals interface806 to facilitate multiple functionalities. For example, motion sensor810, light sensor 812, and proximity sensor 814 may be coupled toperipherals interface 806 to facilitate orientation, lighting, andproximity functions of the device. For example, in some implementations,light sensor 812 may be utilized to facilitate adjusting the brightnessof touch surface 846. In some implementations, motion sensor 810 (e.g.,an accelerometer, rate gyroscope) may be utilized to detect movement andorientation of the device. Accordingly, display objects or media may bepresented according to a detected orientation (e.g., portrait orlandscape).

Other sensors may also be connected to peripherals interface 806, suchas a temperature sensor, a barometer, a biometric sensor, or othersensing device, to facilitate related functionalities. For example, abiometric sensor can detect fingerprints and monitor heart rate andother fitness parameters.

Location processor 815 (e.g., GNSS receiver chip) may be connected toperipherals interface 806 to provide geo-referencing. Electronicmagnetometer 816 (e.g., an integrated circuit chip) may also beconnected to peripherals interface 806 to provide data that may be usedto determine the direction of magnetic North. Thus, electronicmagnetometer 816 may be used as an electronic compass.

Camera subsystem 820 and an optical sensor 822, e.g., a charged coupleddevice (CCD) or a complementary metal-oxide semiconductor (CMOS) opticalsensor, may be utilized to facilitate camera functions, such asrecording photographs and video clips.

Communication functions may be facilitated through one or morecommunication subsystems 824. Communication subsystem(s) 824 may includeone or more wireless communication subsystems. Wireless communicationsubsystems 824 may include radio frequency receivers and transmittersand/or optical (e.g., infrared) receivers and transmitters. Wiredcommunication systems may include a port device, e.g., a UniversalSerial Bus (USB) port or some other wired port connection that may beused to establish a wired connection to other computing devices, such asother communication devices, network access devices, a personalcomputer, a printer, a display screen, or other processing devicescapable of receiving or transmitting data.

The specific design and implementation of the communication subsystem824 may depend on the communication network(s) or medium(s) over whichthe device is intended to operate. For example, a device may includewireless communication subsystems designed to operate over a globalsystem for mobile communications (GSM) network, a GPRS network, anenhanced data GSM environment (EDGE) network, IEEE802.xx communicationnetworks (e.g., Wi-Fi, Wi-Max, ZigBee™), 3G, 4G, 4G LTE, code divisionmultiple access (CDMA) networks, near field communication (NFC), Wi-FiDirect and a Bluetooth™ network. Wireless communication subsystems 824may include hosting protocols such that the device may be configured asa base station for other wireless devices. As another example, thecommunication subsystems may allow the device to synchronize with a hostdevice using one or more protocols or communication technologies, suchas, for example, TCP/IP protocol, HTTP protocol, UDP protocol, ICMPprotocol, POP protocol, FTP protocol, IMAP protocol, DCOM protocol, DDEprotocol, SOAP protocol, HTTP Live Streaming, MPEG Dash and any otherknown communication protocol or technology.

Audio subsystem 826 may be coupled to a speaker 828 and one or moremicrophones 830 to facilitate voice-enabled functions, such as voicerecognition, voice replication, digital recording, and telephonyfunctions.

I/O subsystem 840 may include touch controller 842 and/or other inputcontroller(s) 844. Touch controller 842 may be coupled to a touchsurface 846. Touch surface 846 and touch controller 842 may, forexample, detect contact and movement or break thereof using any of anumber of touch sensitivity technologies, including but not limited to,capacitive, resistive, infrared, and surface acoustic wave technologies,as well as other proximity sensor arrays or other elements fordetermining one or more points of contact with touch surface 846. In oneimplementation, touch surface 846 may display virtual or soft buttonsand a virtual keyboard, which may be used as an input/output device bythe user.

Other input controller(s) 844 may be coupled to other input/controldevices 848, such as one or more buttons, rocker switches, thumb-wheel,infrared port, USB port, and/or a pointer device such as a stylus. Theone or more buttons (not shown) may include an up/down button for volumecontrol of speaker 828 and/or microphone 830.

In some implementations, architecture 800 may present recorded audioand/or video files, such as MP3, AAC, and MPEG video files. In someimplementations, device 800 may include the functionality of an MP3player and may include a pin connector for tethering to other devices.Other input/output and control devices may be used.

Memory interface 802 may be coupled to memory 850. Memory 850 mayinclude high-speed random access memory or non-volatile memory, such asone or more magnetic disk storage devices, one or more optical storagedevices, or flash memory (e.g., NAND, NOR). Memory 850 may storeoperating system 852, such as Darwin, RTXC, LINUX, UNIX, OS X, WINDOWS,or an embedded operating system such as VxWorks. Operating system 852may include instructions for handling basic system services and forperforming hardware dependent tasks. In some implementations, operatingsystem 852 may include a kernel (e.g., UNIX kernel).

Memory 850 may also store communication instructions 854 to facilitatecommunicating with one or more additional devices, one or more computersor servers, including peer-to-peer communications. Communicationinstructions 854 may also be used to select an operational mode orcommunication medium for use by the device, based on a geographiclocation (obtained by the GPS/Navigation instructions 868) of thedevice.

Memory 850 may include graphical user interface instructions 856 tofacilitate graphic user interface processing, including a touch modelfor interpreting touch inputs and gestures; sensor processinginstructions 858 to facilitate sensor-related processing and functions;phone instructions 860 to facilitate phone-related processes andfunctions; electronic messaging instructions 862 to facilitateelectronic-messaging related processes and functions; web browsinginstructions 864 to facilitate web browsing-related processes andfunctions; media processing instructions 866 to facilitate mediaprocessing-related processes and functions; GPS/Navigation instructions868 to facilitate GPS and navigation-related processes; camerainstructions 870 to facilitate camera-related processes and functions;and other instructions 872 for performing some or all of the featuresand processes, as described in reference to FIGS. 1-7.

Each of the above identified instructions and applications maycorrespond to a set of instructions for performing one or more functionsdescribed above. These instructions need not be implemented as separatesoftware programs, procedures, or modules. Memory 850 may includeadditional instructions or fewer instructions. Furthermore, variousfunctions of the device may be implemented in hardware and/or insoftware, including in one or more signal processing and/or applicationspecific integrated circuits (ASICs).

The features described may be implemented in digital electroniccircuitry or in computer hardware, firmware, software, or incombinations of them. The features may be implemented in a computerprogram product tangibly embodied in an information carrier, e.g., in amachine-readable storage device, for execution by a programmableprocessor; and method steps may be performed by a programmable processorexecuting a program of instructions to perform functions of thedescribed implementations by operating on input data and generatingoutput.

The described features may be implemented advantageously in one or morecomputer programs that are executable on a programmable system includingat least one programmable processor coupled to receive data andinstructions from, and to transmit data and instructions to, a datastorage system, at least one input device, and at least one outputdevice. A computer program is a set of instructions that may be used,directly or indirectly, in a computer to perform a certain activity orbring about a certain result. A computer program may be written in anyform of programming language (e.g., Objective-C, Java), includingcompiled or interpreted languages, and it may be deployed in any form,including as a stand-alone program or as a module, component,subroutine, or other unit suitable for use in a computing environment.

Suitable processors for the execution of a program of instructionsinclude, by way of example, both general and special purposemicroprocessors, and the sole processor or one of multiple processors orcores, of any kind of computer. Generally, a processor will receiveinstructions and data from a read-only memory or a random access memoryor both. The essential elements of a computer are a processor forexecuting instructions and one or more memories for storing instructionsand data. Generally, a computer may communicate with mass storagedevices for storing data files. These mass storage devices may includemagnetic disks, such as internal hard disks and removable disks;magneto-optical disks; and optical disks. Storage devices suitable fortangibly embodying computer program instructions and data include allforms of non-volatile memory, including by way of example, semiconductormemory devices, such as EPROM, EEPROM, and flash memory devices;magnetic disks such as internal hard disks and removable disks;magneto-optical disks; and CD-ROM and DVD-ROM disks. The processor andthe memory may be supplemented by, or incorporated in, ASICs(application-specific integrated circuits). To provide for interactionwith a user the features may be implemented on a computer having adisplay device such as a CRT (cathode ray tube), LED (light emittingdiode) or LCD (liquid crystal display) display or monitor for displayinginformation to the author, a keyboard and a pointing device, such as amouse or a trackball by which the author may provide input to thecomputer.

One or more features or steps of the disclosed embodiments may beimplemented using an Application Programming Interface (API). An API maydefine on or more parameters that are passed between a callingapplication and other software code (e.g., an operating system, libraryroutine, function) that provides a service, that provides data, or thatperforms an operation or a computation. The API may be implemented asone or more calls in program code that send or receive one or moreparameters through a parameter list or other structure based on a callconvention defined in an API specification document. A parameter may bea constant, a key, a data structure, an object, an object class, avariable, a data type, a pointer, an array, a list, or another call. APIcalls and parameters may be implemented in any programming language. Theprogramming language may define the vocabulary and calling conventionthat a programmer will employ to access functions supporting the API. Insome implementations, an API call may report to an application thecapabilities of a device running the application, such as inputcapability, output capability, processing capability, power capability,communications capability, etc.

A number of implementations have been described. Nevertheless, it willbe understood that various modifications may be made. Elements of one ormore implementations may be combined, deleted, modified, or supplementedto form further implementations. In yet another example, the logic flowsdepicted in the figures do not require the particular order shown, orsequential order, to achieve desirable results. In addition, other stepsmay be provided, or steps may be eliminated, from the described flows,and other components may be added to, or removed from, the describedsystems. Accordingly, other implementations are within the scope of thefollowing claims.

What is claimed is:
 1. A method comprising: determining, by a device,that the device is attached to a swinging limb based on a sensor signal;determining, by the device, a rotational velocity component of devicevelocity due to the swinging limb based on the sensor signal and a limbposition vector; estimating, by the device, device velocity using aglobal navigation satellite (GNSS) velocity or velocity determined fromwireless signals transmitted by a wireless network; and determining, bythe device, a user velocity by excluding the rotational velocitycomponent from the device velocity.
 2. The method of claim 1, whereindetermining that the device is attached to a swinging limb based on thesensor signal, further comprises: detecting periodicity in the sensorsignal; and determining that the device is attached to the swinging limbbased on the detected periodicity in the sensor signal.
 3. The method ofclaim 1, wherein determining that the device is attached to a swinginglimb based on the sensor signal, further comprises: detecting a firstperiodicity in device acceleration data; detecting a second periodicityin device orientation data; and determining that the device is attachedto the swinging limb based on a relative strength of the first or secondperiodicities.
 4. The method of claim 1, wherein determining arotational velocity component due to the swinging limb based on thesensor signal and limb parameters, further comprises: determining a limbrotation rate vector in a reference coordinate frame; determining aswinging limb position vector in the reference coordinate frame; anddetermining the rotational velocity component based on a cross productof the limb rotation rate vector and the swinging limb position vector.5. The method of claim 4, wherein one or more of the swinging limbparameters are estimated by a device state estimator.
 6. The method ofclaim 4, wherein determining a swinging limb position vector in thereference coordinate frame further comprises: determining terminuspoints of a swing path of the limb; and interpolating between theterminus points to determine the swinging limb position vector.
 7. Themethod of claim 1, wherein one of the swinging limb parameters is a limblength that is obtained from user input.
 8. The method of claim 1,wherein determining user velocity based on the velocity of the deviceand the rotational velocity component uses the formula,

_(user)=

_(device)−

_(rotation)=

_(device)−(

_(L)×

), where

_(user) is the user velocity,

_(device) is the device velocity,

_(L) is a limb rotation rate vector and

is a limb position vector.
 9. The method of claim 1, wherein the uservelocity is used to update a device state estimator between measurementtime epochs.
 10. The method of claim 9, where the update occurs at timeindicated by a trigger signal or schedule based on a periodicity featuredetected in the sensor signal.
 11. A device comprising: one or moresensors; one or more processors; memory coupled to the one or moreprocessors and configured to store instructions, which, when executed bythe one or more processors, causes the one or more processors to performoperations comprising: determining that the device is attached to aswinging limb based on a sensor signal; determining a rotationalvelocity component of device velocity due to the swinging limb based onthe sensor signal and a limb position vector; estimating device velocityusing a global navigation satellite (GNSS) velocity or velocitydetermined from wireless signals transmitted by a wireless network; anddetermining a user velocity by excluding the rotational velocitycomponent from the device velocity.
 12. The device of claim 11, whereindetermining that the device is attached to a swinging limb based on thesensor signal, further comprises: detecting periodicity in the sensorsignal; and determining that the device is attached to the swinging limbbased on the detected periodicity in the sensor signal.
 13. The deviceof claim 11, wherein determining that the device is attached to aswinging limb based on the sensor signal, further comprises: detecting afirst periodicity in device acceleration data; detecting a secondperiodicity in device orientation data; and determining that device isattached to the swinging limb based on a relative strength of the firstand second periodicities.
 14. The device of claim 11, whereindetermining a rotational velocity component due to the swinging limbbased on the sensor signal and limb parameters, further comprises:determining a limb rotation rate vector in a reference coordinate frame;determining a swinging limb position vector in the reference coordinateframe; and determining the rotational velocity component based on across product of the limb rotation rate vector and the swinging limbposition vector.
 15. The device of claim 14, wherein one or more of theswinging limb parameters are estimated by a device state estimator. 16.The device of claim 14, wherein determining a swinging limb positionvector in the reference coordinate frame further comprises: determiningterminus points of a swing path of the limb; and interpolating betweenthe terminus points to determine the swinging limb position vector. 17.The device of claim 11, wherein one of the swinging limb parameters is alimb length that is obtained from user input.
 18. The device of claim11, wherein determining user velocity based on the velocity of thedevice and the rotational velocity component uses the formula,

_(user)=

_(device)−

_(rotation)=

_(device)−(

_(L)×

), where

_(user) is the user velocity,

_(device) is the device velocity,

_(L) is a limb rotation rate vector and

is a limb position vector.
 19. The device of claim 11, wherein the uservelocity is used to update a device state estimator between measurementtime epochs.
 20. The device of claim 19, where the update occurs at timeindicated by a trigger signal or schedule based on a periodicity featuredetected in the sensor signal.